A Randomized Controlled Trial

Background: Concept-based Indexing is purported to provide more granular data representation for clinical records"'2 This implies that a detailed clinical terminology should be able to provide improved access to clinical records. To date there is no data to show that a clinical reference terminology is superior to a precoordinated terminology in its ability to provide access to the clinical record. Today, ICD9-CM is the most commonly used method ofretrieving clinical records. Objective: In this study, we compare the sensitivity, specificity, positive likelihood ratio, positive predictive value and accuracy of SNOMED-RT vs. ICD9-CM in retrieving ten diagnosesfrom a random sample of2,022 episodes ofcare. Method: We randomly selected 1,014 episodes of carefrom the inpatient setting and 1,008 episodes of care from the outpatient setting. Each record had associated with it, the free text final diagnoses from the Master Sheet Index at the Mayo Clinic and the ICD9-CM codes used to billfor the encounters within the episode of care. The free text diagnoses were coded by two expert indexers (disagreements were addressed by a Staff Clinician) as to whether queries regarding one of 5 common or 5 uncommon diagnoses should return this encounter. Thefree text entries were automatically coded using the Mayo Vocabulary Processor. Each of the ten diagnoses was exploded in both SNOMED-RT and ICD9-CM and using these entry points, a retrieval set was generated from the underlying corpus of records. Each retrieval set was compared with the Gold Standard created by the expert indexers. Results: SNOMED-RTproduced significantly greater specificity in its retrieval sets (99.8% vs. 98.3%, p<0.001 McNemar Test). The positive likelihood ratios were significantly better for SNOMED-RT retrieval sets (264.9 vs. 33.8, p<0.001 McNemar Test). The positive predictive value of a SNOMEDRT retrieval was also significantly better than ICD9CM (92.9% vs. 62.4%, p<O.001 McNemar Test). The accuracy defined as 1 (the total error rate (FP+FN) / Total # episodes queried (20,220)) was significantly greater for SNOMED-RT (98.2% vs. 96.8%, p=0.002 McNemar Test). Interestingly, the sensitivity of the SNOMED-RT generated retrieval set was not significantly differentfrom ICD9-CM, but there was a trend toward significance (60.4% vs. 57.6%, p=0.067 McNemar Test). However, if we examine only the outpatient practice SNOMED-RT produced a more sensitive retrieval set than ICD9CM (54.8% vs. 46.4%/, p=0.002 McNemar Test). Conclusions: Our data clearly shows that information regarding both common and rare disorders is more accurately identified with automated SNOMED-RT indexing using the Mayo Vocabulary Processor than it is with traditional hand picked constellations of codes using ICD9-CM. SNOMED-RT provided more sensitive retrievals of outpatient episodes ofcare than ICD9-CM.